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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)^ Responsive to communication(s) filed on 06 October 2005 , 
2a)n This action is FINAL. 2b)S This action is non-final. 

3) n Since this application is in condition for allowance except for fornnal matters, prosecution as to the nnerits is 

closed in accordance with the practice under £x pa/te Quay/e, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-28 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) n Claim(s) is/are allowed. 

6) 13 Claim(s) 1-18,21 and 24-28 is/are rejected. 

7) IE] Claim(s) 19,20,22 and 23 is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10) 13 The drawing(s) filed on 08 November 2001 is/are: a)[3 accepted or b)n objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

11) 0 The oath or declaration is objected to by the Examiner. Note the attached Office Action orfomn PTO-152. 

Priority under 35 U.S.C. § 119 

12) 13 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or(f). 
3)13 All b)n Some * c)\3 None of: 

1 Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 



Specification 

1 . The following guidelines illustrate the preferred layout for the specification of a utility 
application. These guidelines are suggested for the applicant's use. 

Arrangement of the Specification 

As provided in 37 CFR 1.77(b), the specification of a utility application should include 
the following sections in order. Each of the lettered items should appear in upper case, without 
underlining or bold type, as a section heading. If no text follows the section heading, the phrase 
"Not Applicable" should follow the section heading: 

(a) TITLE OF THE INVENTION. 

(b) CROSS-REFERENCE TO RELATED APPLICATIONS. 

(c) STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR 

DEVELOPMENT. 

(d) THE NAMES OF THE PARTIES TO A JOINT RESEARCH AGREEMENT 

(e) INCORPORATION-BY-REFERENCE OF MATERIAL SUBMITTED ON A 

COMPACT DISC (See 37 CFR 1 .52(e)(5) and MPEP 608.05. Computer program 
listings (37 CFR 1.96(c)), "Sequence Listings" (37 CFR 1.821(c)), and tables 
having more than 50 pages of text are permitted to be submitted on compact 
discs.) or 

REFERENCE TO A "MICROFICHE APPENDIX" (See MPEP § 608.05(a). 
"Microfiche Appendices" were accepted by the Office until March 1, 2001.) 

(f) BACKGROUND OF THE INVENTION. 

(1) Field of the Invention. 

(2) Description of Related Art including information disclosed under 37 CFR 1.97 
and 1.98. 

(g) BRIEF SUMMARY OF THE INVENTION. 

(h) BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S). 

(i) DETAILED DESCRIPTION OF THE INVENTION. 

(j) CLAIM OR CLAIMS (commencing on a separate sheet). 

(k) ABSTRACT OF THE DISCLOSURE (commencing on a separate sheet). 

(1) SEQUENCE LISTING (See MPEP § 2424 and 37 CFR 1.821-1.825. A "Sequence 
Listing" is required on paper if the application discloses a nucleotide or amino 
acid sequence as defined in 37 CFR 1.821(a) and if the required "Sequence 
Listing" is not submitted as an electronic document on compact disc). 
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2, The disclosure is objected to because of the following informalities: The specification 
fails to label each section appropriately. 

Appropriate correction is required. 

Claim Rejections - 35 USC §101 

3. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or 
any new and useftjl improvement thereof, may obtain a patent therefor, subject to the conditions and 
requirements of this title. 

Claim 27 is rejected under 35 U.S.C. 101 because the claimed invention is directed to 
non- statutory subject matter. Claim 27 is drawn to functional descriptive material NOT claimed 
as residing on a computer readable medium. MPEP 2106.IV.B. 1(a) (Functional Descriptive 
Material) states: 

"Data structures not claimed as embodied in a computer-readable medium are descriptive 
material per se and are not statutory because they are not capable of causing functional change in 
the computer." 

"Such claimed data structures do not define any structural or functional interrelationships 
between the data structure and other claimed aspects of the invention which permit the data 
structure's functionality to be realized." 

Claim 27, while defining a "computer program", does not define a "computer-readable 
medium" and is thus non-statutory for that reasons. A "computer program" can range from 
paper on which the program is written, to a program simply contemplated and memorized by a 
person. The examiner suggests amending the claim to embody the program on "computer- 
readable medium" in order to make the claim statutory. 
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"In contrast, a claimed computer-readable medium encoded with the data structure 
defines structural and functional interrelationships between the data structure and the computer 
software and hardware components which permit the data structure's functionality to be realized, 
and is thus statutory " - MPEP 2106.IV.B.l(a) 



Claim Rejections - 35 USC § 112 
4, The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

Claims 1-13 are rejected under 35 U.S.C. 1 12, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which applicant regards as 
the invention. 

Claim 1 recites the Umitation "said interactive content data" in lines 18-19. There is 
insufficient antecedent basis for this limitation in the claim. 



Claim Rejections - 35 USC §103 

5, The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
maimer in which the invention was made. 

6. Claims 1-5, 8, 1 1-18, 21 and 24-28 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over US 5,590,262 to Isadore-Barreca in view of US 5,708,845 to Wistendahl et al. 



("Wistendahl"). 
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As to claim 1, Isadore-Barreca discloses an apparatus for associating data representative 
of an object with a digital video programme including: 

Means for providing a digital video programme having plural individual frames at least 
some of which incorporate said object (column 5, lines 49-52)t, 

Means for parsing the video programme by identifying separate shots in the video 
programme to produce an edit Ust (column 6, lines 33-61), 

means For identifying shots containing related content to form a sequence of shots 
containing related content (column 6, lines 33-61, wherein in and out points define sequence 
boundaries and a sequence corresponds to a scene or series of shots with related content), 

means for selecting at least one key-frame from within each sequence of shots (column 7, 
lines 31-35), 

means for selecting said object within the key-frame with which data is to be associated 
(column 9, lines 51-58, wherein hot spots correspond to object within the key- frame), 

means for extracting attributes of the object from the key-frame (column 10, lines 35-38, 
wherein hot spot coordinates correspond to attributes of the object), 

means for associating interactive data with the object in the key- frame (column 9, lines 
13-17 and column 10, lines 50-53), 

whereby said interactive data is associated with the object in the sequence of shots and 
said interactive content data is embedded with data representative of said object in a data 
sequence (column 9, lines 13-17 and column 10, lines 50-53). 

Isadore-Barreca does not disclose expressly tracking means for utilizing the attributes of 
the object for tracking the object through the sequence of shots. 
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However, Wistendahl discloses an apparatus for associating data representative of an 
object with a digital video programme including: 

means for selecting said object within the key-frame with which data is to be associated 
(column 11, lines 25-35), 

means for extracting attributes of the object from the key-frame (column 1 1, lines 41-44, 
wherein N Data corresponds to attributes of the object), 

means for associating interactive data with the object in the key- frame (column 1 1, line 
66 - column 12, line 8), 

tracking means for utilizing the attributes of the object for tracking the object through the 
sequence of shots (column 11, lines 3-44), 

whereby said interactive data is associated with the object in the sequence of shots and 
said interactive content data is embedded with data representative of said object in a data 
sequence (column 6, lines 21-39). 

Isadore-Barreca & Wistendahl are combinable because they are from the same art of 
image processing, specifically associating interactive data with an object in a video sequence. 

At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to provide the apparatus disclosed by Isadore-Barreca with the tracking means as taught 
by Wistendahl. 

The suggestion/motivation for doing so would have been that the motion tracking tool 
saves the author from having to draw the outline around the object in each frame of the sequence 
(Wistendahl, column 1 1, lines 20-22). 
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Therefore, it would have been obvious to combine Isadore-Barreca with Wistendahl to 
obtain the invention as specified in claim 1 . 

As to claim 2, the combination of Isadore-Barreca and Wistendahl disclose an apparatus 
as claimed in claim 1, wherein the means for identifying shots containing related content to form 
a sequence of shots containing related content includes means for producing a hierarchy of 
groups of shots (Isadore-Barreca, column 7, line 51 - column 8, line 19). 

As to claim 3, the combination of Isadore-Barreca and Wistendahl discloses an apparatus 
as claimed in claim 1, wherein said means for parsing include means for inputting criteria to be 
used to recognize a change of shot (Isadore-Berreca, column 6, lines 47-61, in and out points 
correspond to criteria used to recognize a change of shot). 

As to claim 4, the combination of Isadore-Barreca and Wistendahl discloses an apparatus 
as claimed in any of claims 1, wherein the means for extracting attributes of the object includes 
means for isolating the object within a boundary formed on the frame, means for performing 
edge detection within the boundary to identify and locate edges of said object, and storing means 
for storing a geometric model of said object (Isadore-Barreca, column 9, lines 62-67 and column 
10, lines 35-38). 

As to claim 5, the combination of Isadore-Barreca and Wistendahl discloses an apparatus 
as claimed in claim 1, wherein said means for extracting attributes of said object also includes 
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means for recording at least one of the attributes of shape, size, position, colour, texture, statistics 
based on said attributes (Isadore-Barreca, column 10, lines 35-38, coordinates correspond to 
position). 

As to claim 8, the combination of Isadore-Barreca and Wistendahl disclose an apparatus, 
as claimed in claim 1, wherein the means for utilizing the attributes of the object for tracking the 
object includes means for updating the stored attributes of the object as the attributes of the 
object change from frame to frame (Wistendahl, column 11, lines 3-24). 

As to claim 1 1, the combination of Isadore-Barreca and Wistendahl disclose an apparatus 
as claimed in claim 1, wherein the means for associating interactive data with the object in the 
key-frame includes means for providing a database of different types of data including one or 
more of URLS, HTML pages, video clips, audio clips, text files and multimedia catalogues, and 
means for selecting said interactive content data from the database to associate with said object 
(Isadore-Barreca, column 1 1, lines 10-15 and Wistendahl column 6, lines 21-39). 

As to claim 12, the combination of Isadore-Barreca and Wistendahl discloses an 
apparatus as claim in claim 1, wherein the means for associating interactive data with the object 
in the key-frame produces said data sequence using means for determining whether the 
embedded interactive content data is frame synchronous data associated with object positions, 
shapes, ranks and pointers in a frame, or group-synchronous data associated with all the objects 
in a group, or is data to be streamed just in time, wherein means for provided for associating 
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frame synchronous data with the corresponding frame, means are provided for associating group 
synchronous data with the frame at which a group changes, and means are provided for 
streaming just in time data to a user before it is required to be associated with the corresponding 
objects (Isadore-Barreca, column 11, lines 10-15 and Wistendahl column 6, lines 21-39). 



As to claim 13, the combination of Isadore-Barreca and Wistendahl disclose an apparatus 
as claimed in any of the preceding claims, wherein means are provided to associate different 
interactive content data with respectively different objects (Isadore-Barreca, column 1 1, lines 10- 
15 and Wistendahl column 6, lines 21-39). 



As to claim 14, please refer to 
As to claim 15, please refer to 
As to claim 16, please refer to 
As to claim 1 7, please refer to 
As to claim 18, please refer to 
As to claim 21, please refer to 
As to claim 24, please refer to 
As to claim 25, please refer to 
As to claim 26, please refer to 



the rejection of claim 1 above, 
the rejection of claim 3 above, 
the rejection of claim 2 above, 
the rejection of claim 4 above, 
the rejection of claim 5 above, 
the rejection of claim 8 above, 
the rejection of claim 1 1 above 
the rejection of claim 12 above 
the rejection of claim 13 above 



Application/Control Number: 09/889,966 Page 10 

Art Unit: 2625 

As to claim 27, the combination of Isadore-Barreca and Wistendahl disclose a computer 
program comprising code means for performing all the steps of the method of claim 14, when the 
program is run on one or more computers (Isadore-Barreca, column 12, lines 1-7). 

As to claim 28, the combination of Isadore-Barreca and Wistendahl disclose a computer 
program as claimed in claim 27, wherein the computer program is embodied on a computer- 
readable medium (Isadore-Barreca, column 12, lines 1-7). 

Allowable Subject Matter 

7. Claims 19, 20, 22 and 23 are objected to as being dependent upon a rejected base claim, 
but would be allowable if rewritten in independent form including all of the limitations of the 
base claim and any intervening claims. 

8. Claim6, 7, 9 and 10 would be allowable if rewritten to overcome the rejection(s) under 35 
U.S.C. 1 12, 2nd paragraph, set forth in this Office action and to include all of the limitations of 
the base claim and any intervening claims. 

Conclusion 

9. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

USPN 6,205,23 1 to Isadore-Barreca et al. discloses an interactive video system. 



Application/Control Number: 09/889,966 



Page 1 1 



Art Unit: 2625 

USPN 6,357,042 to Srinivasan et al, discloses an interactive video system. 
USPN 6,449,608 to Morita et al. . discloses an interactive video system, 

10. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Aaron W. Carter whose telephone number is (571) 272-7445. 
The examiner can normally be reached on 8am - 4:30 am (Mon. - Fri.). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Jingge Wu can be reached on (571) 272-7429. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
appUcations is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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